Where a storage apparatus incorporates a great number of devices such as HDDs (Hard Disk Drives) ready for the SAS (Serial Attached SCSI) which is one of the SCSI (Small Computer System Interface) standards, a SAS Expander (hereinafter referred to as EXP) is sometimes used.
The EXP is a SAS switch used for expanding a SAS connection.
FIG. 7 is a view illustrating an example of a configuration of a storage system including a plurality of SAS devices 700 and EXPs 400 and 600. Particularly, FIG. 7 illustrates an example of a large-scale storage system which uses a great number of SAS devices 700 and EXPs 400 and 600.
As illustrated in FIG. 7, the storage system includes a storage apparatus 100 including a controller enclosure (hereinafter referred to as CE) 200 and a plurality of drive enclosures (hereinafter referred to as DEs) 500, and a host 800.
It is to be noted that, in FIG. 7, a double line indicates a host interface and a single line indicates a SAS connection.
The CE 200 carries out control of a data access request for writing and readout of data and so forth from the host 800 to the SAS devices 700, and includes a controller module (hereinafter referred to as CM) 300 and an EXP 400.
The CM 300 is a module that manages subordinate SAS devices 700 and allocates an appropriate SAS device 700 to a data access request from the host 800 such as a server to a SAS device 700. Further, the CM 300 functions as an initiator of the SAS devices 700 and the EXPs 400 and 600 and carries out access to a SAS device 700 that is an access target through the subordinate EXPs 400 and 600.
The EXP 400 is a module for expanding a SAS connection and includes an expander chip that is a SAS controlling device and a Phy that is a physical port of the SAS.
Further, the EXP 400 is connected to the CM 300 and the EXP 600 in the subordinate DE 500 through the Phy and transfers a data access request from the CM 300 to the EXP 600 that is an access target.
It is to be noted that, in order to achieve redundancy, the CE 200 includes a plurality of (eight in the example illustrated in FIG. 7) CMs 300 and has a multi-initiator configuration. Further, in order to achieve redundancy, the CE 200 includes a plurality of (two in the example illustrated in FIG. 7) EXPs 400.
The DE 500 incorporates a plurality of SAS devices (SAS drives) 700 such as HDDs therein and includes a plurality of (two in the example illustrated in FIG. 7) EXPs 600 in order to achieve redundancy. It is to be noted that, in FIG. 7, while illustration of some SAS devices 700 and illustration of reference numerals 600 and 700 in some DEs 500 is omitted for simplified illustration of the figure, all of the DEs 500 have a similar configuration.
It is to be noted that, in a data path from a CM 300 to a DE 500, the CM 300 side and the DE 500 side are hereinafter referred to as upper side and lower side, respectively.
The EXP 600 has a configuration similar to that of the EXPs 400 in the CE 200 and is connected to the EXP 400 in a CE 200 and/or an EXP 600 in a different DE 500 through a Phy (physical port) not illustrated. The EXP 600 carries out a process in accordance with a data access request from the upper side, that is, from the CM 300 side.
It is to be noted that, when it is necessary to specify one of the CMs 300 in the following description, one of CMs 00 to 07 is used as illustrated in FIG. 7. Further, when it is necessary to specify one of the EXPs 400 or 600 in the following description, one of EXPs 00, to 17, 20 to 27, 30 to 37 and 40 to 47 is used as illustrated in FIG. 7.
As illustrated in FIG. 7, in the storage apparatus 100, the CMs 00 to 07 are connected to the upper side of the EXPs 00 and 01. Further, the EXPs 10 to 13 are connected to the lower side of the EXP 00 while the EXPs 14 to 17 are connected to the lower side of the EXP 01.
Further, the EXPs 20, 30 and 40 are cascade-connected (multistage-connected) in series to the EXP 10; the EXPs 21, 31 and 41 to the EXP 11; EXPs 22, 32 and 42 to the EXP 12; and the EXPs 23, 33 and 43 to the EXP 13. Similarly, the EXPs 24, 34 and 44 are cascade-connected (multistage-connected) in series to the EXP 14; the EXPs 25, 35 and 45 to the EXP 15; the EXPs 26, 36 and 46 to the EXP 16; and the EXPs 27, 37 and 47 to the EXP 17.
Further, the two EXPs 600 in each DE 500 are connected to at least one SAS device 700 included in the DE 500 to which the EXPs themselves belong.
The storage apparatus 100 configured in such a manner as described above can connect a great number of SAS devices 700 using the EXPs 400 and 600.
Here, the EXPs 400 and EXPs 600 are provided two by two in each enclosure for redundancy. In the redundant EXPs 400 and 600, the series in which the EXP 00 in the CE 200 and each of the EXPs 10 to 13 in the DEs 500 are connected to each other is hereinafter referred to as first series, and the series in which the EXP 01 in the CE 200 and each of the EXPs 14 to 17 in the DEs 500 are connected to each other is hereinafter referred to as second series.
It is to be noted that the EXPs 10 to 13, EXPs 20 to 23, EXPs 30 to 33 and EXPs 40 to 43 of the individual systems in the first series are hereinafter referred to collectively as EXPs 10 to 43.
Incidentally, if some state variation occurs with one of the subordinate SAS devices 700 of the EXPs 10 to 43 of the first series in the configuration of FIG. 7, then the EXP 600 to which the SAS device 700 is connected issues a broadcast change (hereinafter referred to as BC (CHG)) to the upper side. It is to be noted that the state variation of a SAS device 700 may be, for example, insertion and pullout of a SAS device 700 or device disorder.
The BC (CHG) is a notification indicating occurrence of a state variation and is transferred to all of the CMs 300 through the EXPs 600 and 400 on the upper side from the EXP 600 that is an issuance source.
The CM 300 that is an initiator carries out, after reception of the BC (CHG), a discovery process for re-recognizing the subordinate SAS devices 700.
It is to be noted that, since the EXPs 400 and 600 receiving the BC (CHG) transfer the BC (CHG) to all of the SAS ports, that is, to all of the Phys, the BC (CHG) is propagated to the upper side, and as a result, all of the CMs 300 start a discovery process at a substantially same timing.
In the discovery process, the CM 300 issues a plurality of SMP (SAS Management Protocol) Functions to all of the EXPs 400 and 600 connected to the lower side. The SMP Function is a command defined by the SMP which is a protocol for managing the EXP, and is used for acquisition of information of the subordinate SAS devices 700 of the EXP, construction of routing information of the EXP, construction of topology, control of the Phys, link negotiation for the SAS connection and so forth.
The CM 300 issues five to ten SMP Functions (commands), for example, to each of the EXPs 400 or the EXPs 600 in order to acquire information intended for the discovery process.
The EXPs 400 and 600 receiving the SMP Function issue information corresponding to a kind of the command, for example, information of the number of the Phys possessed thereby and the SAS devices 700 connected to the Phys, as map information to the CM 300.
The CM 300 manages the information of the SAS devices 700 on the lower side using the map information transmitted thereto from the EXPs 400 and 600 on the lower side.    [Patent Document 1] Japanese Laid-Open Patent Publication No. 2006-279347    [Patent Document 2] Japanese Laid-Open Patent Publication No. 2009-540436
As described above, in the discovery process, the CM 300 issues a plurality of SMP Functions to all of the EXPs 400 and 600.
On the other hand, the EXPs 400 and 600 can carry out processes regarding the received plural SMP Functions only one by one.
Accordingly, in the multi-initiator environment illustrated in FIG. 7, there is a subject that, when the discovery process occurs, accesses by the SMP Functions from the plural CMs 300 conflict and a high load is applied to the EXPs 400 and 600.